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Abstract 


In line with technological development, the computer-based national examination (CBNE) has become 
an urgent matter as its implementation faces various challenges, especially in developing countries. 
Strategies in implementing CBNE are thus needed to face the challenges. The aim of this research was to 
analyse the challenges and strategies of Indonesian schools in implementing CBNE. This research was 
qualitative phenomenological in nature. The data were collected through a questionnaire and a focus 
group discussion. The research participants were teachers who were test supervisors and technicians at 
junior high schools and senior high schools (i.e. Level 1 and 2) and vocational high schools implementing 
CBNE in Yogyakarta, Indonesia. The data were analysed using the Bogdan and Biklen model. The results 
indicate that (1) in implementing CBNE, the schools should initially make efforts to provide the electronic 
equipment supporting it; (2) the implementation of CBNE is challenged by problems concerning the 
Internet and the electricity supply; (3) the test supervisors have to learn their duties by themselves 
and (4) the students are not yet familiar with the beneficial use of information technology. To deal with 
such challenges, the schools employed strategies by making efforts to provide the standard electronic 
equipment through collaboration with the students’ parents and improving the curriculum content by 
adding information technology as a school subject. 

Keywords: computer-based national examination, challenges in implementing CBNE, strategies for 
implementing CBNE. 


Introduction 


Each level of schools needs to conduct an evaluation in order to know the extent of the 
learning objective is attained. This evaluation is not only conducted by the school but also 
by the government. One of evaluation forms conducted by the Indonesian government is the 
national examination for students at the final year of each level of education. According to 
the Regulation of the Minister of Education and Culture of the Republic of Indonesia No.5 
Year 2015, the national examination is the activity of nationally measuring and evaluating the 
attainment of passing competence in a certain school subject (Mendikbud, 2015). This national 
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examination also has a role in sustaining the educational process as an important instrument for 
evaluating students’ competence (Bichi & Musa, 2015). Nitko and Brookhart (2011) state that 
the occurrence of an evaluation process is employed to obtain information to use to make a 
decision related to students, curriculum, learning programs, and educational policy. Therefore, 
the national examination as one of the evaluation forms in education has a very important role 
in education aspects around the world. Particularly in Indonesia, the national examination today 
no longer serves as the sole determining factor in student graduation, but rather, it serves to map 
the quality of the learning program and/or the school as stated in the Government Regulation of 
the Republic of Indonesia No.13 Year 2015. It is widely accepted that the results of the national 
examination can give a clear picture about the reality of education in Indonesia including the 
programs being run and other things related to educational systems. This educational mapping 
can help portray the condition of education in Indonesia in more detail and thus some follow- 
up actions can be further taken to improve it. This is in line with what has been explained by 
Bichi and Musa (2015) that evaluation in education is a process of collecting information of 
students’ learning and performance to improve education, help measure students’ competence 
in acquiring certain behaviour or specific knowledge of certain topics, and become the basis for 
ascertaining the quality of education at all levels. 

However, the implementation of the previous national examination in Indonesia is 
inefficient and time-consuming. A lengthy process has been required to determine the results 
of the exam. Meanwhile, the development of this new era along with the development of 
technology today also have a significant impact on administering the national examination. 
The great demand for quick results has urged the need for technology utilization in it. The 
utilization of technology for the national examination in Indonesia can make the execution 
more effective, particularly in test distribution to schools in regions far from the city centre. 
The technological advancement for the national examination in Indonesia is then realized in 
the form of Computer-Based National Examination (CBNE) or also known as the national 
examination of Computer-Based Test (CBT) system. CBNE is a computer-based test whose 
item selection and presentation are administered in a computerized manner in which the test 
takers are given a different package of test items each (BSNP, 2015). In this test, students will 
no longer use a paper-based version, but rather, they completely use a computer-based system 
already programmed to get the results faster. 

Indonesia began to administer the CBNE in the academic year of 2014/2015 
(Mendikbud, 2015). It was only conducted by particular schools in some regions that fulfilled 
the predetermined criteria for administering CBNE. The schools administering CBNE at this 
initial stage are called CBNE-pilot schools. Today, more schools of junior high school (JHS) 
and senior high school (SHS) levels (Level 1 and 2) and vocational high schools (VHS) have 
already used the CBNE system. The limited number of schools administering CBNE is due to 
the reality that not all schools fulfil the predetermined requirements or criteria. According to 
the Agency of National Educational Standards (BSNP, 2015), the schools administering the 
national examination with a computer-based system have to previously submit a statement 
to confirm their willingness to participate as a test executor. The next step is checking by a 
verification team to make sure that the schools are well-equipped with facilities and resources 
needed. 

Concerning technical requirements, schools administering CBNE are required to provide 
computers with the minimum specifications for both server and client computers (main and 
reserve), an Internet connection network with a minimum bandwidth of 1 Mbps and a Local 
Area Network (LAN). The server computer (main and reserve) devices are PC/tower/desktop 
(not laptop); accessible port 80; 4 core processor with a frequency of 1.6 GHz; the RAM is 8 
GB at the minimum; the operating system (of 64 bytes) is Windows Server 8/Windows 7/Linux; 
there are two units of LAN CARD; the UPS (Uninterruptible Power Supply) which can last 
for 15 minutes; the number of server computers is according to the ratio of 1:40 (1.e., 1 server 
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computer is for 40 client computers, at the maximum); and there is at least one reserve server 
computer. Meanwhile, the main and reserve client computers should meet several requirements 
as follows: PCs or laptops with, at the minimum, monitors of 12 inch, processors of one core 
with a frequency of | GHz; a RAM of 512 MB; the operating system is, at the least, Windows 
XP installed as .NET Framework 3.5; the web browser is, at the minimum, Chrome with copied 
Exambro (the browser application from the educational evaluation centre for CBNE); the 
minimum number of client computers is according to the ratio of 1:3 (1.e., 1 client computer 
for 3 participants); the reserve client computers are at least 10%; and headsets/earphones are 
provided (for the listening examination at senior high school). For the LAN hardware, the 
requirements are cables at least CATSE 10/100/1000; each server with | switch and at least 
24ports; a bandwidth of 1 Mbps dedicated; the IP made static (the determination of the IP being 
made by the central executor); and at least 1 reserve unit. Besides, in general, UPS at least for 
server for 2 hours long and a generator set (or genset) for all the electronic equipment used in 
CBNE should be provided (Kemendikbud, 2017). 

Schools that offer themselves to administer CBNE should fulfil the criteria not only 
related to computer facilities but also related to human resources. They should have at least one 
test supervisor, one technician, and room supervisors as needed who fulfil the criteria stated 
in the standard operational procedure (SOP) of the national examination. The criteria for test 
supervisors are as follows: they are teachers or educational personnel at a school who possess 
competence in information and communication technology; they have attended a relevant 
training or have served as a test supervisor in CBNE; they are willing to be assigned as test 
supervisors at schools administering CBNE; and they do not refuse to sign a pact of integrity. 
The criteria for technicians are as follows: they are teachers or educational personnel who 
have knowledge, skill, and experience in managing the school LAN; they have attended a 
relevant preparatory program or have served as a CBNE technician; and they are willing to sign 
a pact of integrity. Meanwhile, the criteria for supervisors are as follows: they are disciplined, 
honest, careful and responsible teachers who are able to keep something in confidentiality; they 
are in a healthy condition and able to supervise a national examination; they are not teachers 
of the subjects being supervised in the examination; they are not from the schools of CBNE 
participant; and they are willing to sign a pact of integrity (BSNP, 2017). 

Although there is an increasingly greater number of schools that implement CBNE every 
year, the implementation still faces various problems. In general, the geographical condition 
of Indonesia becomes a crucial factor which can trigger those problems. As widely known, 
Indonesia consists of thousands of islands with diverse natural landscapes including the 
lowlands near the coast, the highlands and also large areas of hills stretching across regions 
in Indonesian. Indeed, the diversity of Indonesian geographical features can be a constraint in 
implementing CBNE. For example, there are some challenges in the provision of computers, 
internet connection and other facilities required for CBNE implementation. The weather 
condition that is occasionally unpredictable, such as heavy rain with thunder and lightning 
occurring in the daytime when CBNE is in progress, can disrupt or even damage the computer 
system and internet connection. As a developing country, Indonesia has not maximized 
technology utilization yet in many aspects of life. Internet connectivity is also a big challenge 
here because it is still unevenly distributed to all over regions in Indonesia, especially remote 
areas. Besides the constraints of internet access, the stability of electricity supply also becomes 
another challenge in administering CBNE (Sanni & Muhammad, 2015). Similarly, the stability 
of the electricity in Indonesia is also limited, especially in remote regions. Another constraint 
of CBNE implementation in Indonesia is the social condition of its society. People in particular 
regions in Indonesia do not completely have awareness about the importance of education for 
their life, and thus, do not really care about matters supporting their children’s education. 

Regardless the various constraints faced in administering CBNE, there are some results 
of research comparing a paper-based national examination with a computer-based national 
examination. A study conducted by Retnawati (2015) indicates that a paper/pencil-based test 
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and a computer-based test generally do not differ significantly in results and both of them do not 
influence the characteristics of the testing medium used. In addition, results of a study conducted 
by Piaw (2012) show that a computer-based national examination can be used to replace a 
paper-based national examination in testing at educational institutions. Similar findings are also 
given by Horchlehnert, Brass, Moeltner, & Juenger (2011) through a study which reveals that 
a computer-based examination and a paper-based national examination for students of health 
sciences give the same result, by implication, a computer-based examination can be used as 
a good method to do written examination efficiently and fairly. Furthermore, a psychometric 
test using a computer-based model also gives the same result as that using the paper/pencil- 
based model (Baumer, Roded, & Gafni, 2009). The advantages of computer-based testing are: 
it is efficient; it improves students’ self-efficacy and motivation (Chua, 2012), and students have 
positive attitude toward CBT (Dammas, 2016). 


Research Aim 


Along with the development of this technological era, the various constraints in 
administering CBNE are expected to be well overcome and anticipated. It is because CBNE is 
necessary to be conducted even though its implementation still faces various challenges at the 
moment. The research concerned here was therefore particularly to analyse the challenges and 
strategies of Indonesian schools in the CBNE implementation. 


Researcher Role 


In this research, the researchers acted as observers during the CBNE implementation. 
They investigated the phenomena of challenges faced by schools and their strategies in 
implementing CBNE. By using a questionnaire and a focus group discussion, the researchers 
collected the data, recorded them in field notes and then analysed them. The researchers then 
made the conclusion based on their understanding of the existing phenomena. 


Methodology of Research 
Type of the Research 


This research is qualitative phenomenological in nature. The type of this research 
is descriptive-exploratory research. The phenomenon described in this research was the 
challenges faced by JHSs, SHSs, and VHSs in implementing CBNE. The challenges in the 
CBNE implementation were examined from several points including the equipment, the 
human resources, the test takers (students), and the environment. This research also analysed 
the strategies used by the schools in implementing CBNE so that the examination could run 
smoothly. 


Time and Scope of Research 


This research was conducted from February to August 2017. It involved schools 
implementing CBNE in the Special Territory of Yogyakarta. There were two school levels 
both public and private schools including: (1) Junior High Schools (JHS) and (2) Senior High 
Schools (SHS) and Vocational High Schools (VHS). These schools were representative of 
schools located in urban and rural areas. 
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Data and Participants 


The data were collected by using a focus group discussion (FGD). The FGD was held 
one time. Before the FGD was held, the participants were required to fill in an open-ended 
questionnaire. This questionnaire included a number of items related to the sets of hardware 
(the server, the reserve server, and the client computers), the topology of a computer network, 
stability of the Internet connection, availability of electricity supply, the application setting 
(program installation, synchronization with the central system, and the running of Exambro), 
troubleshooting when administering CBNE, checking whether the students’ answers had been 
already sent to the central server, the preparedness of the test supervisors, technicians, and the 
participants (students) of CBNE try-out and implementation, and the security for test equipment 
for several shifts of CBNE implementation. The questionnaire items were developed according 
to the requirements needed for the CBNE implementation synthesised from the theoretical 
review. The participants’ responses to the questionnaire were then discussed in the FGD. In- 
depth interviews were also held when further clarification about the FGD data was needed. 

Using the data obtained from the Educational Department of Yogyakarta about schools 
implementing the CBNE, 45 schools were invited to join the FGD. For each educational level, 
there were 15 schools including 10 public schools and 5 private schools located in urban and 
rural areas. There was one test supervisor and one technician for each school. Among 45 
schools, there were 35 schools attending the FGD, consisting of 35 test supervisors (P1-P35) 
and 33 technicians (T1-T33) as the research participants of this study. They came from 11 
JHSs, 12 SHSs, and 12 VHSs in Yogyakarta. The participants had at least one-year experience 
in implementing the CBNE at their schools (consisting of four series of activities, i.e. three 
simulations and one real CBNE implementation). 


Data Analysis 


The data obtained from the FGD were qualitative in nature consisting of various challenges 
and strategies in implementing the CBNE. The data were recorded in field notes and analysed 
through data reduction. The results were then presented in tables and re-examined to identify 
their subthemes. Afterwards, the data were analysed using a model developed by Bogdan & 
Biklen (1982) to determine the inter-theme relations and gain more detail understanding. The 
results of this analysis were used to understand the phenomenon of challenges and strategies in 
implementing the CBNE in JHSs, SHSs and VHSs. 


Ethical Considerations 


The researcher informed the FGD participants of this research which was conducted to 
obtain their opinions about the CBNE implementation and strategies to overcome the challenges. 
This information can be used in other test locations. It can be also used to capture improved 
inputs, especially for educational policy makers in order to implement CBNE much better. To 
maintain the credibility of the data in this study, the name of the schools and the participants 
(test supervisors and technicians) involved in this study had been identified with pseudonyms, 
and all the data had been kept confidential as part of the research ethics procedures. 


Results of Research 
The implementation of CBNE at a number of schools in Indonesia still faces various 


constraints. However, the schools can use various strategies to overcome the constraints. The 
challenges and strategies are explained in detail as follows. 
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Challenges of CBNE Implementation 


In 2017, all JHSs and VHSs in Yogyakarta had already implemented CBNE. To implement 
CBNE, the schools must have adequate facilities so that the computer-based examination 
system could run well. However, facts in the field indicated that many schools were faced 
with constraints in their efforts to fulfil the requirements of facilities which meet the minimum 
standards determined by the government. The mapping of challenges in facility requirements 
for CBNE implementation found in this research are presented in Table 1. 


Table 1. Challenges in fulfilling facility requirements for CBNE. 





Challenges Theme Inter-Theme Relation 


The server computers are expensive. 





Many schools do not yet have 
ideal server computers as 
recommended by BSNP. 


Some schools do not have standard server computers 
yet. 





The number of server computers is limited. 





The computers in the school laboratory are already Tisumneeanaeecnel The problems related to 
old-fashioned. @ number and specitication —_the availability of comput- 


—____ - of client computers become ers which do not meet 
The specifications of the client computers do not meet constraints in administering the standardsand linited 
the standards. ; : 

; eS CBNE. rooms influence the ratio 

The number of client computers is limited. of server computers to 
The room area is limited. client computers, the 
Limited space influences the placement layout of 
ratio of the server computers computers, and the cal- 
to the client computers and culation of the electricity 
The computers in the school laboratory are perma- the ideal computer layout. power required. 
nently positioned. 














It is difficult to realize the ideal distance from one 
computer to the other. 





A great quantity of electricity power is required. 
There is difficult access to electricity in particular 
schools. 

There are rolling blackouts. 





Many schools are constrained 
by electricity power and 
stability. 








The availability of computers fulfilled the standards, sufficient rooms, and electricity 
become the main assets that should be possessed by the schools in implementing CBNE. 
However, the fact is that there are many constraints encountered by the schools in providing 
the aforementioned three components as required. The first constraint is the availability of 
computers. The school should provide computers as the server and client with the minimum 
specifications as follows: the server computer is a PC/tower/desktop in type (i.e., not a laptop) 
with a Xeon or 15 processor, a RAM of 8 GB, a hard disk of 250 GB, and a UPS while the 
client computer is a PC/laptop with a monitor of 12 inch in screen size, a dual core processor, 
and a RAM of 512 MB. Another requirement is that 1 server computer accommodates 40 client 
computers at most, there is at least one reserve server computer, one client computer could be 
used by 3 participants at most, and there are reserve client computers of at least 10% in quantity. 
Ideally, the need for server and client computers can be met by utilizing the computers already 
available in the laboratories at the schools. However, the fact is that the majority of schools 
do not have the computers with the specifications required and even if they have them, their 
number could not match the number of CBNE participants needing them. The cost of server and 
client computers is a heavy burden for the school because not only one computer unit is needed 
to serve as a client computer. Rather, the number of the client computer units should be equal 
with the number of the CBNE participants. Consequently, there are many schools which do 
not have computers with the minimum specifications and numbers required for administering 
CBNE. 
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The second constraint is the availability of rooms. In some cases, the schools do not 
have the building specifically provided for administering CBNE. Generally, the schools make 
use of a laboratory or a classroom, but this strategy raises some problems. The first problem 
is that the room area does not conform to the design for administering CBNE. This aspect is 
crucial because it is closely related to the ratio of the client computers to the server computers. 
The distance from one computer to the other is well arranged in order that participants cannot 
communicate and see one another’s monitor screen (BSNP, 2017) or the distance is at least 1 
meter (Kemdikbud, 2017). The limited server computer ownership makes the schools wish to 
be able to administer CBNE with a server-client computer ratio of 1:40 in order that the number 
of the server computers needed is as minimum as possible. However, having | server and 40 
client computers in a room with the ideal distance from one computer to the other definitely 
requires a very spacious room. According to the test supervisors, the laboratory commonly used 
as the place for administering CBNE at schools could only accommodate 20 up to 36 client 
computers on average. The second problem is related to the placement layout of computers. In 
general, the schools make use of the computer laboratory to administer CBNE. Unfortunately, 
the computers in the laboratory at some schools are permanently installed in fixed positions 
and it is impossible to modify the layout. Therefore, the requirement for providing a classroom 
seating arrangement model with no seat facing each other as exhorted by Kemdikbud (2017) 
could not be realized at these schools. 

The implementation of CBNE requires a good sufficient electric installation. Turning 
on very large numbers of electronic devices simultaneously requires a great stable electric 
power. Mostly the schools, which implement CBNE for the first time, attempt to upgrade 
their electricity power in order that all the electronic devices turn on effectively. If the client 
computers provided by the school use tube monitors, the electricity needed is extremely greater. 
Another problem with the electricity occurs at some schools located on the provincial borders 
because the access to electricity is relatively difficult. A very common trouble is a temporary 
shutdown of electric power (or rolling blackout). This problem is a major worry for the school. 
If such a blackout occurs when CBNE is being conducted, it will disrupt the process because all 
the equipment used is based on electronic devices with electricity supply from PLN. 

Stability of Internet networks and understanding of Exambro software greatly determine 
whether CBNE runs smoothly. Table 2 presents the mapping of the challenges in the provision 
of Internet networks and Exambro software. CBNE requires a good and stable Internet 
connection. However, the fact is that even in Yogyakarta province, which is categorized as a 
signal-rich zone, there are still regions whose stability in Internet connection is not sufficiently 
conducive. According to a test supervisor, the Internet connection at his school tends to be slow 
so that it is not sufficiently conducive for administering CBNE. To improve the quality of the 
Internet connection, the school already makes efforts to propose the installation of networks 
based on optical fibres but until CBNE is held, it has not been yet realized. School location also 
influences the stability of the Internet connection. There are schools that are geographically 
located in a mountainous region which is lightning-prone so that there is a great risk of network 
damage to occur, as proven when the Internet connection was down during CBNE try out 
because the device for Internet connection was hit by lightning. 
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Table 2. Challenges in providing internet networks and running the Exambro. 619 








Challenges Theme Inter-Theme Relation 





The geographical location of the school is not suf- 
ficiently conducive. Many schools are constrained by 
unstable Internet access. 





The network at some schools is not stable. 





The committee at 
school should know 
the potentials of the In- 


The Exambro updating is too close to the day of 


CBNE implementation. The committee at school should 
al ee sites Le feraokconnecton-and 
in order that the software runs well. understand Exambro 


Windows OS does not all give support. well so that everything 
could run smoothly 





Exambro cannot run compatibly with all browsers. 








The central system is at times down. 


When listening tasks are fo be done, no sound is There are constraints caused by the 
heard. central source. 








Exambro could be minimized. 





The constraints in operating the digital system in CBNE are divided in terms of two 
sources, namely, the school local system and the central system. The constraints whose source 
is the school local system are related to the OS (Operating System), the Exambro updating 
process, and the web browser. The OS is the house of all the applications in the computer. 
Using the newest OS version turns out to be not a guarantee of maximum performance of each 
computer unit in CBNE. Each OS type or version has its own character, especially in relation 
with specifications, in order to be able to run well. Some schools choose the OS Win10, which 
is the newest OS version. At a JHS, however, the installation of Win10 in server computers did 
not result in good collaboration with the CBNE system. The school decided to downgrade the 
OS server to Win7. With the older OS version, it turned out that the server computer could run 
well. 

Exambro is the software used in administering CBNE. The software is updated in 
accordance with a request from the central server so that the local server gets the newest version. 
The problem that arises is that Exambro requires updating so often and its newest version 
usually releases very closely to the timing of CBNE implementation. At an SHS, for example, 
the Exambro updating was conducted closely to the day of CBNE implementation and this 
greatly disturbed the preparation. The test supervisors and technicians had to work overtime to 
complete the preparation and make sure that all the systems ran well with the newest Exambro 
version. 

The last constraint coming from the school local system is the choice of web browser. 
As stated in the manual of CBNE implementation (Kemdikbud, 2017), Exambro would be 
optimum in performance if using the library application web browser ‘Google Chrome.’ 
However, in practice, many schools were often constrained in choosing the web browser. A 
test supervisor at a VHS stated that some schools had a problem concerning the web browser 
because the version of Google Chrome they installed is the newest one (i.e., too sophisticated). 
Choosing this newest Google Chrome version turned out to be useless as it did not to give a 
good impact and it rather became incompatible with the system (troublesome). 

The next source of constraints came from the central system including the system 
performance, the set of examination items, and the working system of Exambro. These 
constraints could not be controlled by the local committee. It is true that the central system 
is the controller of all the systems running at the schools, but the implementation of CBNE, 
which is simultaneously conducted and involves so many participants, definitely raises the 
risk of disturbance in the central system. If the central system is down, the CBNE process will 
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620| be interrupted or even totally stopped. As a result, the implementation of CBNE did not run 
effectively as planned. In practice, various problems related to the central system occurred 
at the schools in administering CBNE. In an SHS for example, there was a day of CBNE 
implementation when the system was down due to problems from the central server. On the 
third day, the system was even offline, and therefore, the CBNE implementation was delayed by 
65 minutes. Another SHS also faced a system trouble from the central server and consequently 
had to endure a 30-minute delay in administering CBNE. However, it is quite possible that at 
one school, those system constraints only partially occur. 

The set of examination items is an absolute element that the local committee cannot 
control when any problem arises. All test items are managed by the central system and given 
to the local committee a few moments before CBNE is administered. A problem is found at an 
SHS when the listening section of English could not run as it should be. This constraint became 
worse as it only happened to six of the client computers in the room. 

The last matter related to the performance of the system that the test supervisor and 
technician complain about is the Exambro software page appearance that could be minimized 
on the monitor screen. Such application ability to be minimized gives an impact that is not so 
good to the process of administering CBNE because it enables the students to simultaneously 
make use of another application in the computer. The case occurring most frequently is the 
utilization of the calculator-making application (which comes in a built-in condition with the 
computer) to aid the calculation process in answering a test item. According to the examination 
regulations, however, students are forbidden to use such a calculating aid. 

Students’ readiness in participating in CBNE is one of the aspects determining the 
smoothness in administering CBNE. However, the data indicate that there are still various not- 
so-ideal students’ conditions that make the administering of CBNE become hampered. The 
results of the mapping of CBNE constraints whose source is the student are as presented in 
Table 3. 

The CBNE system makes differences sufficiently significant mainly because the process 
of doing the examination is under computer control. To be able to do the items well, students 
should first be familiar with the basic points of operating the computer, which is followed 
with understanding the related software and specifically the Exambro software. The students 
participating in CBNE vary in condition, there being those who are already IT literate and those 
who are still IT ignorant. Therefore, the school to administer CBNE arranges an introduction to 
CBNE by administering a try-out. However, data from technicians and test supervisors mention 
that students are not all competent yet at CBNE time. The main cause of the insufficient 
competence in IT is that not all students have the computer facility at home and not all students 
make use of the try-out well as means of introducing to them the system of the computer-based 
examination. 





Table 3. Readiness of CBNE participants/students. 





Inter-Theme 


Readiness of CBNE Participants/Students Theme : 
Relation 





Students are not sufficiently skilled in utilizing the computer Students’ mastery of information Siudaniearenat 











for doing the examination when taking part in CBNE. technology and particularly of vel alkeble'ts 
Exambro is still insufficient. ; 

Students are surprised by the strict timer system in CBNE. Dene oe Pseeres adjust them- 

Students make pranks and fool around in making use of the selves to CBNE 
Some students do not feel and students with 

Students’ motivation is insufficient. sufficient responsibility for them- special needs 





selves and the CBNE hardware. are still con- 
strained in taking 
Many students are allergic to coldness. There are constraints related to part in CBNE. 
CBNE could not yet sufficiently facilitate the disabled. health and special needs. 


Students are not sufficiently disciplined. 
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Another problem related to students’ readiness for the system concerns the time 
allocations of the examination. In the paper-and-pencil examination system used in the UN or 
other examinations, the time controller is the examination supervisor who is a teacher while 
the time control in CBNE is done by the computer system. The students are not yet familiar 
with the accurate time allocations because other examinations like final term examination or 
midterm examination are paper-and-pencil-based so that there is emotional interference from 
the supervisor often resulting in the occurrence of extra time. The students look surprised and 
unprepared when the time for working in the CBNE is ended by the system. 

Students’ sense of responsibility also becomes a focus of the spotlight from the school’s 
authorities. Some cases indicate that often students do not feel sufficiently responsible for the 
CBNE equipment that they use and for themselves. Some schools complain about there being 
students making use of CBNE equipment not as they should; in fact, some test supervisors call 
it a prank or fooling around, which is intentionally amusing oneself by doing something else. 
The mouse and the headset are two computer accessories that students usually play with so that 
some of them are damaged. Such behaviour is considerably exhibited by students from another 
school which hitches a ride in taking the CBNE because their school does not have sufficient 
facilities. It is indicated that such behaviour appears because there is no sense of the objects as 
those belonging to them (i.e., the objects are not facilities belonging to their own school). 

As for students’ responsibility to themselves, many of them are still insufficiently 
motivated to do the CBNE. A test supervisor at an SHS reported the extreme case of some 
students coming to do the examination items for 15 minutes only. It was also seen that the 
discipline among them was not so good. The students didn’t do the test seriously, as indicated 
by their time when they had done it. 

Besides IT and responsibility, in the execution of CBNE, some students are constrained 
by the health factor. The CBNE is generally held in the laboratory with the air conditioner (AC) 
facility running and it makes students who are allergic to coldness become uncomfortable. In 
general, the reaction in the part of the allergic students is a blocked nose, a headache, a feeling 
of nausea, or a very severe cold feeling. With their body in such conditions, students are greatly 
hampered in their efforts to be able to do the examination items to the maximum. 

The next thing concerns students’ basic abilities. In 2017, CBNE still could not 
facilitate the disabled well. Some inclusive schools held a special paper-and-pencil-based 
national examination for their disabled students. The main reason concerned disabled students’ 
preparedness because the schools were not yet able to get their disabled students ready for 
participation in CBNE and the disabled students themselves did not yet have sufficient self- 
confidence to do computer-based examination items. For the students with disabled eyesight, 
there was another reason: in CBNE, items with pictures were not complemented with spoken 
description of the pictures so that sightless students could not get accurate access to them. 

The final kind of constraint in administering CBNE is from the committee and supervisor 
sector. There are various situations that are not sufficiently ideal so that many complaints 
arise from test supervisors, and technicians. Results of the mapping done to the constraints 
encountered by test supervisors and technicians from the preparation until the execution of 
CBNE are as in Table 4. 
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Table 4. Constraints encountered by the CBNE committee and supervisors. 





Constrains Theme Inter-Theme Relation 





The number of teachers who were prepared to become 
technicians and test supervisors is at a minimum. There is a lack of human 
resources who are qualifie 





d The brevity of the prepara- 











The number of teachers who understand IT is ata mini- in IT. tion program and minimum 
mum. quantity of human resources 
who are qualified in IT make the 
The preparation given to test supervisors and technicians understanding of those on duty 
is insufficient. 7 of the system and execution of 
Test supervisors’ and technicians’ understanding is not yet The preparation given to CBNE insufficient. The problem 
fini technicians and test supervi- ig worsened with the addition 
sors is not yet maximum. —_ of the stamina being sapped 
The mastery of the CBNE SOP is insufficient. because CBNE is held in three 
shifts. 





Many schools administer CBNE in 3 shifts. The a Sopeisnteralid 
technicians are exhausted. 

The initial constraint experienced by the school is the selection of the personnel to 
function as a technician and a test supervisor. These two positions demand sufficient IT insight 
in the individuals concerned in order that they could do their job well. Some schools admit 
having great difficulty in seeking the personnel with such competence; in fact, because there 
are no teachers and other school employees who are prepared to become technicians, a JHS is 
forced to recruit technicians from sources external to the school. 

The constraints experienced by test supervisors and technicians are initiated with the 
admission that their understanding of CBNE is still not yet firm and often there are still doubts 
at times of being confronted with a case. Theoretically, such a situation ought not to occur 
because they have been given some briefing. However, according to their confession, the 
briefing received is still quite a minimum so that there are still many matters that they still could 
not thoroughly understand. Often, they encounter a problem that has not been mentioned at the 
time of the preparation and orientation program so that they could not respond to it as constraint 
speedily and accurately. 

In administering CBNE, a constraint that often arises is difference in opinion between 
a test supervisor and a committee concerning task distribution. Basically, the tasks are already 
assigned in the SOP of the national examination, but it seems that some personnel do not 
thoroughly understand yet their respective tasks. In a case occurring at a VHS, the task of being 
responsible for the participants’ presence list was passed to a test supervisor while it ought to 
be the responsibility of a committee member. Besides the problem of SOP mastery, committee 
members, technicians, and test supervisors also have difficulties in maintaining their stamina, 
especially at schools administering CBNE up to 3 shifts (with Session | from 07.30 a.m. to 
09.30 a.m., Session 2 from 10.30 a.m. to 12.30 p.m., and Session 3 from 1.30 p.m. to 3.30 
p.m.). The majority of the supervisors, technicians, and test supervisors complain about being 
exhausted when they are on the job in the last session. 


Solution Strategies 


Budget limitations are among the main causes of the school’s difficulties in dealing with 
the various shortcomings of the facilities, especially in relation with matters concerning the 
availability of computers meeting prescribed standards and the electricity. For overcoming 
problems in the midst of budget limitations, the school takes various alternative steps in order 
that CBNE could keep running smoothly though without the support of ideal funding. Results of 
the mapping done for the strategies for overcoming the challenges resulting from the limitations 
of the school funding are presented in Table 5. 
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Table 5. Strategies for overcoming the challenges. 623 








Strategies Theme Inter-Theme Relation 





Using non-standard server 
Some schools make use of available 


computers as servers and do some 

borrowing 

Borrowing server from students’ parents Budget limitations force 
the school to be smart in 
making use of available 
assets and maximize 
their use via collaboration 
through partnership 


Modifying the computer so that it is appropriate 
to become a server 








Borrowing client computers To provide client computers, the 
school make them available, bor- 
row them, and upgrade laboratory 
Buying client computers computers 





Upgrading laboratory computers 








Arranging computer layout according to the 
available room space Adjusting computer layout to room 
size and maximizing hardware use 





Improving the quality of the connector cable 





The price of local server computers for CBNE that is quite expensive is a problem for 
the school. Viewed from the point of their specifications, not all schools have computers with 
such requirements. Supplying them is the way most easily thought of to overcome the problem 
of server availability. However, not all schools have the financial ability to supply themselves 
with such computers by purchasing them. Such a situation demands a solution that takes the 
school’s financial stability into consideration. Many schools make modifications on their 
available computers to make them fit for their new use. Some computers with specifications of 
a lower level are combined to produce a new one with specifications approaching those newly 
required and then it is perfected by means of supplying it with new hardware parts to make 
it meet the new standard of at least the minimum level. The schools admit that the strategy is 
sufficiently effective to provide servers with specifications at the minimum level of meeting the 
new needs at minimum cost. The second strategy is using a server with specifications lower in 
level than those of the minimum level recommended by the government. That strategy is used 
by some schools because they are already unable to find anymore possible source of funding. 
The decision to use a server with specifications below the standard set is surely highly risky for 
the smooth running of CBNE. Therefore, the school makes use of the try-out time to test server 
performance and it turns out that some schools using servers with processors of lower levels 
could administer CBNE smoothly. The third strategy is borrowing the server needed. However, 
in view of the relatively high level of the server specifications required, not all schools have 
access to such borrowing since very rarely are there people already having computers of such 
specifications and, moreover, the CBNE server should not be a laptop. 

The Ministry of Education and Culture issues the decision that one client computer 
could be used by three students in the sense that CBNE could be held in three shifts per day. 
If the school has 270 students participating in CBNE, it should provide 100 client computers 
consisting of 90 core computers plus 10 reserve computers. The main problem concerning the 
availability of client computers is the availability in accordance with the number needed. In the 
majority, the schools admit that they are not yet able to meet that need independently. The first 
strategy used to meet the need is making the computers available by buying them. However, 
the very great number needed makes it impossible for the school to meet all the needs through 
the lane of purchases. The second strategy is modifying old computers until they are fit to be 
used as client computers. The system used to do it is generally cannibalism, namely, combining 
fitting components from various computers until they become one package fit to become a 
client computer. In addition, the system of upgrading the specifications of old computers is 
also used until they are fit to be client computers in CBNE. Some computers that are already 
old and with specifications under the standards set are identified and then the parts not yet 
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meeting the minimum standards set are upgraded. The third strategy is the final one because it 
is used only when one feels very much forced to do it. It is the strategy of borrowing a laptop 
or PC from various sources, especially teachers and other employees of the school. If the loans 
from teachers and other school employees still do not meet the need yet, the school generally 
collaborates with students’ parents or guardians. In the majority, schools combine the three 
strategies to meet the needs. 

Room availability is a problem which is relatively difficult to solve because it is not 
something that could be provided all of a sudden and its provision also takes very great expense. 
The absence of a special room for CBNE forces the school to make use of a classroom equipped 
with adequate electric supply, Internet access, and space area. Generally, the school takes the ICT 
laboratory as one chosen room because it has adequate Internet connection and electric power 
already with the addition of one or more other rooms with electric, Internet, and areal settings 
according to need. During CBNE, a constraint that could not be controlled to conform to the 
ideal condition is the inter-computer distance. The school tends to arrange the positional layout 
of anumber of computers in accordance with room area though the ideal distance is 1 meter and 
the exhortation concerning computer positional layout that the computers do not face each other 
could not be complied with yet. The school puts aside the matter of distance and endeavours 
to catch up with the number of participants, considering that the CBNE items are already in 
random order when presented to each participant so that the possibility for participants to copy 
each other’s work is relatively small. To maximize the usefulness of the available computer 
positional layout, generally the school makes efforts to use purchased connector cables of the 
best quality. 

Electricity is the source of energy that should be well prepared in order that CBNE runs 
smoothly. Various constraints concerning the process of providing electric current is anticipated 
in various ways by the school. Table 6 presents the results of some mapping of the strategies in 
responding to constraints in supplying energy sources in administering CBNE. 

The school endeavours to prepare the energy sources as well as possible in order 
that the CBNE process runs smoothly, making available client computers varying in power 
consumption like laptops, PCs with LCD/LED monitors, and PCs with tube monitors, which 
require considerable energy the most. To cope with the various needs of electricity, some electric 
circuit manipulation (or power distribution) is done so that each room could be supplied with 
electricity as needed. 


Table 6. Strategies for supplying electricity at schools. 





Inter-Theme Rela- 











Strategies Theme : 
tion 
Manipulating electric circuits 
Collaborating with the State Electric Company Renewal and maintenance of electric 
: mn circuits and preparation of reserve Preparing, manag- 
Increasing the power of electricity energy source ing, and arranging 





strategies for maxi- 
mizing the electric 
Focusing on the electric consumption for CBNE potential possessed 
Being focused, effective, and ef- 

ficient in using the electricity 


Provision of a generator set 








Minimizing the use of electronic equipment 





A strategy to guarantee the electric supply needed is collaborating with the State Electric 
Company, especially concerning maintenance of electric circuits. In general, schools have a 
power quota that is more than enough for daily needs, but it would be disturbed when the use 
approaches the limits set. Some schools admit that the power quota for them ought to be able 
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to start the electronic apparatus needed but in practice the electric power is always down when 
used to simultaneously start all the electronic devices. The main cause is the rare checking 
done to the performance of the circuits. In such cases, collaboration with the State Electric 
Company becomes crucial in the course of maximizing the electric potential in order to be 
able to calculate accurately the difference between the power quota and the amount of power 
needed so that if indeed the power supplied is not yet enough, then power raising is done. In 
some regions that are often subject to scheduled rotation of blackouts, the school makes efforts 
to conduct certain communication with the State Electric Company in order not to be subjected 
to scheduled blackout at the time of CBNE execution. In addition, to guarantee surety of energy 
source, the school provides itself with a genset. 

Besides efforts of getting provided with energy sources, the school also makes 
economization of the electric consumption at CBNE time. All the electric power is focused 
on the room where CBNE is held. Various electronic apparatus outside the CBNE room with 
no need to be turned on are shut down. Hopefully, in that way there is no disturbance from 
the electric power being down due to the electricity being absorbed by electronic instruments 
outside the CBNE room. The next strategy is minimizing the electric consumption inside the 
CBNE room. Some schools turn off various electronic equipment not affecting the CBNE 
system, especially the AC, in order that the electric consumption is at a minimum and the 
electric power does not become down. 

CBNE is a digitally-based examination system that depends on the stability of the Internet 
connection and the performance of the application planted in each hardware. Facts in the field 
indicate that the school with its local committee should do extra work in order to be able to 
provide a stable network and assurance of smoothness in the running of all applications. Table 
7 presents results of the mapping done for the strategies for providing the Internet connection 
and the participants’ responses to the software-related constraints. 


Table 7. Strategies for providing the network and the participants’ responses to 
constraints related to the Exambro application. 























Strategies Theme Inter-Theme Relation 
Preparing the modem Solution to network problems 
= depends on the school’s ability 
Shee in funding. Solution to network stability 
Installing the update using the administrative constraints depends on 
remote funding ability supported 
Working overtime for the test supervisors and Constraints related to the with the work ethic, creativ- 
technicians Exambro system are overcome _—"*s - maturity ns 
; ae i on duty in responding to 

Exchanging computers during listening section with the work ethos, creativity, 

ae 9 9 and maturity of the committee in Problems during the execu- 
Communicating with the test centre dealing with the constraints. tion of CBNE. 





Managing students’ psychical condition 


Generally, the school makes efforts to provide the network for Internet connection which 
is based on optical fibres because it is considered the most able to provide Internet connection 
with good stability. However, it is unfortunate that not all schools are already able to get an 
administrator of such a network. The next alternative for the school to take if not having any 
access to optical fiber network is providing itself with a tower. The reason is that the tower- 
based network is considered better and more stable in performance compared to a modem. 

To assure network availability, the school generally provides itself with, at the minimum, 
two types of network. The majority of schools opt for the modem as stand-by companion of 
the main network already prepared. That step is taken to anticipate the occurrence of error in 
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the main network as in a case taking place at a vocational school located in a lightning-prone 
region. Prior to CBNE, the Internet connection network at the school dropped because it was hit 
by lightning and the network was damaged. In that situation, the modem became the solution to 
the problem of providing the Internet connection network. 

One of the characteristics of a modem, compared to another network type, is its relatively 
insufficient sensitivity. Therefore, in the course of preparing a modem, the school should make 
some research on the match between the location and the cellular provider. The strength and 
stability of the signal from each provider vary greatly according to location. In fact, it is quite 
possible that inter-location difference of only around 5 meters could result in highly significant 
difference in Internet performance. An example of an extreme case took place at a vocational 
school which had to provide a modem with three different providers because each room had a 
compatibility with a different provider. 

The next constraint is related to a condition which is not ideal in the Exambro software. 
The first complaint from the school concerns the software updating activity approaching the 
day of administering CBNE. Updating Exambro close to the day of administering CBNE 
is considered disturbing the preparations very much by the school local committee because 
the technicians and test supervisors have to prepare the computers all over again to fit the 
newest Exambro version. The majority of schools inform that, to face that matter, the school 
local committee have to work overtime to prepare the server and client computers one by one. 
However, some schools state that that is not a problem because they use remote admin to do 
the process of installing the newest software. The following are statements from the schools in 
relation with the Exambro renewal. 

The next complaint is about an item-related error that could not be put under control 
by the school committee. A case of this occurred when 6 client computers in one of the rooms 
at an SHS could not present listening items as they should. The test supervisors responded 
to it by doing some manipulated moving of students’ seats. The six students who could not 
do the listening items because of the error were asked to do the reading items first and, after 
finishing it, they were asked to exchange seating positions with six other students who had 
done the listening items. Though taking some moments, that solution was considered effective 
in overcoming the problem. 

The time when the system does not run as it should makes the committee and the 
participants nervous. The first problem found at a JHS was the difficulty to log in. The committee 
hypothesized that the problem resulted from the crowded current due to simultaneous logging 
in. The problem was overcome when the committee instructed the CBNE participants to log 
in in turns based on seating position. The second problem concerned the system becoming 
offline as found at many schools. Generally, the committee recommends that students refresh 
the system by repeating the logging in. In some cases, the strategy runs well but in other cases 
caused by the central server having a problem, the strategy does not give any effect. Then it is 
demanded that the committee makes active communication with the central committee so that 
the real condition related to the constrained system could be identified well. 

The national examination is a very great and important undertaking, particularly for the 
students. Therefore, the occurrence of constraints during the execution of CBNE could make 
them nervous. There is a demand that the test supervisors be able to manage the students’ 
psychic state when encountering technical constraints in order that their mental state remains 
well protected so that they are able to do the examination items with smoothness. The school 
prepares the strategy of managing the students’ psychic state at the time constraints occur during 
the briefing before administering CBNE. 

The constraints faced by the students are important aspects that should be paid attention 
to in order that CBNE runs effectively. In this case, the schools employed various strategies 
to anticipate the challenges. An exposition of the strategies of minimizing the constraints 
originating in the students’ condition during the execution of CBNE is presented in Table 8. 
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Table 8. Strategies for anticipating CBNE constraints faced by the students. 627 
Strategies Theme Interethetae ele: 
tion 

Giving an orientation session Giving stricter orientation and 
dhaiidcesivariick supervision to minimize the students’ Building a good 

9g Pp negative attitudes communication with 
Allowing students to wear their alma mater coats Conditioning the room temperature  CBNE participants/ 

to create a conducive atmosphere Students 

Setting the room temperature for comfort for the students 





The school gives orientation to the students in order that they participate in CBNE 
as well as possible. The orientation emphasizes that the students are to be responsible for 
themselves and for the facilities in the CBNE room. The orientation is done continuously until 
a few minutes before starting to administer CBNE. The orientation is loaded with motivation 
by clearly outlining the functions of CBNE at the level of the students’ personal self, the 
institution, up to the educational mapping mission. Therefore, hopefully, though the national 
examination is no longer the sole determinant of their graduation, they remain in possession 
of high motivation in participating in the national examination. In addition to motivation, the 
orientation also emphasizes using the computers and the accessories in the CBNE room as they 
should be used. A sense of the facilities used as their own is planted in students participating in 
CBNE at their own school and a sense of being responsible for the facilities used is planted in 
students who are visitors at the school where they participate in CBNE. A moment prior to the 
execution of CBNE, students are exhorted to be careful with various matters that could cause 
disturbance like a seated position with legs straightened or attractively posed so that it has the 
potential of damaging/shaking/severing a cable. In concord with the orientation strategy, during 
the execution of CBNE, stricter supervision is made. The strict supervision is done to anticipate 
various students who do not receive the orientation favourably so that their behaviour is still 
not sufficiently decent. 

The next students’ internal problem concerns the room temperature. Students’ responses 
in relation with temperature vary greatly, there being students who like it cold with some others 
being allergic to coldness instead. The test supervisor attempts to adjust the temperature in 
accordance with comfort instead of wish. Efforts are made to enable all students to participate 
well in CBNE without being constrained by allergy. Some students who could not adjust 
themselves to the majority of students’ comfort in the room are allowed to put on their jackets. 
The majority of schools urge that the jacket worn be the uniform coat of the students’ alma 
mater so that students’ neatness and uniformity are constantly maintained. The following are 
some statements by the school in relation with the strategy. 

The constraints coming from the local committee are anticipated by forming a solid 
collaboration as a team. Reduced data of the various actions made by the local committee as 
efforts to neutralize constraints from the preparation until the execution of CBNE are presented 
in Table 9. 
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628] Table 9. Strategies for dealing with constraints faced by the test supervisors 


and technicians. 














Strategies Theme Inter-Theme Relation 
Learning independently or via social media 
group Technicians and test supervisors carry 

out an exploration independently Technicians and test 


Using trial and error : 
supervisors must be 


The personnel, particularly technicians,  'T Competent, in good 
health, independent in 


iti i ici should understand IT 
Recruiting more outside technicians leaming and they should 


Choosing young personnel Competent technicians and test supervi- work together solidly in 
—________________—_— sors with good a stamina who are able a team. 


to work in a team should be intentionally 
chosen 





Choosing persons with IT knowledge 








Maintaining a good cooperation between the 
test supervisor and the technician 


Insufficient possession of knowledge is a hindrance initially felt by test supervisors 
and technicians. The briefing program is felt to be quite minimum in preparing them so that 
it could not yet meet the requirements for the knowledge needed by technicians and test 
supervisors in overcoming various problems. Technicians and test supervisors respond by 
learning independently or in groups after the briefing program. Independent learning is done 
by reading various literature and attempting to overcome problems by using the technique of 
trial and error. Group learning is done through discussions via social media. Thus, problems not 
overcome yet at the time of independent learning could be used as material for discussion in the 
social media group. Various problems are explored in the social media group by test supervisors 
and technicians. 

In order to be able to understand the job fast, those chosen to be on duty should have 
superior IT knowledge. Thus, hopefully those on duty could make use of their already possessed 
IT knowledge as an asset in order to be more competent in responding to various situations. 
If the human resources at the school are not sufficient, the alternative is to recruit IT expert 
personnel from outside the school. Another matter related to personnel is that the school is well 
aware that the execution of CBNE would sap considerable energy because there are very many 
preparations like making sure that all equipment functions are well responding to instructions 
of conducting software update or to regulations from the central system, and coordinating with 
various parties so that the majority of schools choose those young in age and with good stamina 
to be the persons on duty to anticipate moments when they should do extra work and work 
overtime. 

In the execution of CBNE, the persons on duty are to have their own respective jobs but 
in practice they are often overwhelmed by their tasks. Therefore, those on duty are directed to 
be flexible in playing their respective parts and assist each other in doing their job. The strategy 
employed by an SHS is that in the preparation for CBNE, all those on duty become technicians 
while at the time of CBNE all of them become test supervisors. All of them are responsible not 
only for their own respective jobs but also for the accomplishment of CBNE on the whole. The 
strategy is considered sufficiently effective in overcoming the great work load in a relatively 
short time. 


Limitations 


Although the Yogyakarta is a province in Indonesia which has characteristics of the 
region that represents Indonesia culturally and geographically, but in the reality, the regions in 
Indonesia are more varied. Some provinces consist of islands and mountains that lead to the 
increasingly complex challenges of CBNE implementation. Similarly, the ability of students in 
using information technology are also varied and these phenomena certainly affect the CBNE 
implementation. 
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Discussion 


Computer-Based National Examination (CBNE) is one of the evaluation forms done by 
the Indonesian government. It has been annually conducted in Indonesia since a few years ago 
and even the number of the schools implementing it has been continuously increasing. This 
CBNE is conducted to improve the quality of education in Indonesia. Bichi and Musa (2015: 
290) support that the statement by saying that evaluation in education is a process of gathering 
information of students’ learning and performance to improve education and to become the 
basis for ascertaining the quality of education at all levels. It is widely argued that CBNE 
is an effective mass evaluation means in education (Temitayo, Adebisi & Alice, 2013) and 
that each student gets different examination items in CBNE (BSNP, 2015). Hence, CBNE is 
expected to improve the integrity of the national examination. However, it does not mean that 
the implementation of CBNE is already perfect and it does not have any constraint related 
to the equipment for administering it or the personnel involved in it. According to Thurlow, 
Lazarus, Albus, & Hodgson (2010), besides its advantages, a computer-based test also has 
some drawbacks. A number of constraints that occur in the CBNE implementation, particularly 
in Yogyakarta, based on the results of FGD are constraints in providing the CBNE facilities 
including the provision of computers and sufficient rooms which meet the standards and stable 
supply of electricity; constraints in Internet networking (due to geographical location and 
limited network), and constraints in running Exambro; constraints in the CBNE participants’ 
readiness; and constraints from the CBNE committee and supervisors. However, some of these 
constraints could be overcome or anticipated by the school personnel and other parties through 
some efforts. These challenges are in line with the results of studies conducted by Sanni & 
Muhammad (2017) and Abubakar & Adaebayo (2014). 

The constraints in the CBNE implementation related to the provision of various facilities 
including the standard computers and rooms and stable supply of electricity are all very closely 
related with funding. It is also stated by Thurlow, Lazarus, Albus, & Hodgson (2010) that in the 
CBNE implementation, some schools may not possess a sufficient number of computers and 
other supporting types of infrastructure. An alternative solution to deal with the constraint is that 
the schools should precisely utilize the available assets to minimize the lack of computers and 
rooms needed for implementing CBNE. In addition, cooperation with another school partner 
can be done by a particular school in the provision of computers or other equipment needed. 

The next constraint is related to the instability of Internet connectivity in some regions 
where the schools implemented CBNE. This problem is not completely due to the fault or 
unpreparedness of the schools in implementing CBNE, but there are also other influencing 
factors such as the geographical location of the schools. The schools in urban areas could access 
the Internet easily while some other schools within rural areas had difficulty in getting stable 
Internet access. In fact, the Internet access has a very important role in the CBNE implementation 
to connect client computers with the central system such as in the process of delivering the 
students’ answers. The constraint that might occur related to the Internet connectivity could 
be overcome by using a modem with various providers adjusted to the network stability at the 
school location and particularly in each room used for CBNE. In addition, for the schools which 
have the allocation of funds for the provision of Internet network could build a network tower 
to support the process of administering CBNE. 

The process of running Exambro also becomes another constraint particularly for the 
technicians and the committee involved in administering CBNE who do not have sufficient 
experience with and knowledge of the Exambro software. The Exambro updating which usually 
releases very prior to the exam day is considered troublesome for some schools, especially 
for the administrator and committee. However, some other schools that have anticipated this 
constraint appear to be better prepared by making use of administrator remote to adjust the 
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setting of the client computers to the newest updated Exambro. In general, the constraint in 
using the Exambro software can be anticipated by giving adequate preparation and training to 
the CBNE administrator and committee. Some orientation sessions to improve creativity and 
fast response in dealing with Exambro changing or updating also need to be given to them. 
In addition, the administrator, technicians and the committee of CBNE need to improve their 
work ethic and self-autonomy, for example, in learning from various sources about the CBNE 
implementation, particularly related to the software used for CBNE. This is very important 
because the role of the administrator or technician in the CBNE implementation is crucial from 
the beginning (in the preparation) until the end of the exam (such as performing the logout step 
if the students have finished their exam but forget to click the logout button on the computer, 
uploading all the students’ answers to the central server, and other related tasks (Kemendikbud, 
2017). 

There are also constraints coming from the students. The student preparation in facing 
CBNE not only includes their cognitive aspects but also their psychological condition and 
their readiness related to the technical procedure in CBNE which are different from those in 
the previous paper-based national examinations. According to Ackerman and Barnett (2005), 
readiness is often interpreted as a skill, behaviour, and character of the child in relation with the 
expectation of the class or the school. Meanwhile, Sulistyaningsih and Sugiman (2016) state 
that students’ readiness and anxiety have a negative correlation which means that the more the 
students are ready to face the national examination, the less anxious they become. Furthermore, 
the national examination has an impact on teachers and students’ anxiety (Woolfolk & Nicholich, 
1984) and students’ readiness and motivation (Astuti & Retnawati, 2017). The teachers’ anxiety 
is not only related to the students’ competence and readiness in facing CBNE but also the exam 
process, especially in the schools where CBNE is implemented for the first time. Therefore, both 
the teachers and the students need to be given adequate preparation and training to face CBNE. 
As stated by Thurlow, Lazarus, Albus, & Hodgson (2010), in implementing a computer-based 
test, a considerable amount of time is needed for conducting a training for the students about 
the technical procedures of the CBNE system, particularly for those who have no skill in using 
a computer. It is also possible that some students experience anxiety when using a computer in 
their exam (Thurlow, Lazarus, Albus, & Hodgson (2010). The students’ anxiety can impede their 
concentration (Mayer, 2008; Vitasari, Wahab, Othman, & Awang, 2010) and this condition will 
negatively affect their ability in completing CBNE. The school and the student parents in this 
case have an important role in preparing their mental condition and skills in using a computer. 
However, today there is no longer information and communication technology (ICT) subject at 
JHS and SHS levels. In Curriculum 2013, information and communication technology (ICT) 
is omitted from the list of school subjects taught to students. Indeed, this makes students less 
familiar with and capable of using a computer, particularly those studying at remote schools. 

The impact is that many students still have difficulties in using basic parts of computer 
hardware such as keyboards and headsets. The students’ low ability in using keyboards causes 
them to feel more comfortable and well prepared with the paper-based examination than the 
computer-based examination (Al-Amri, 2007). In fact, the presence of ICT as a school subject 
indirectly gives assurance of better facilities and infrastructures from the government such as 
the availability of electricity power and computer laboratory at each school. The ICT subject can 
also become the basis for computer use for students, especially those attending remote schools. 
While the government is currently developing the implementation of CBNE, many students 
in some remote schools are not computer literate. By implication, school officials or teachers 
should work very hard to give their students guidance in using a computer as needed for the 
CBNE implementation. It is expected that the removal of ICT subject from JHS and SHS levels 
will be substituted with the integration of computer use in all school subjects. However, not all 
teachers master computer use. Many teachers also scarcely use a computer in their teaching 
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and learning process. Therefore, this ICT integration will also face some constraints which 
then impede the students’ readiness for participating in CBNE. As widely argued, respondents’ 
unfamiliarity in using technology is one of obstacles in implementing the computer-based 
examination (Retnawati, 2015; Rezaie & Golshen, 2015; Magdin & Turéani, 2015). 

Another constraint in administering CBNE is related to the CBNE committee and 
supervisors. The committee and supervisors in administering CBNE should meet various 
requirements predetermined by the government. Not only the committee in administering 
CBNE is responsible for the process of administering CBNE but also, they should know what 
action would be taken if a problem arises when administering CBNE. Therefore, the committee 
also generally need training first as a step in preparation to administering the computer-based 
test (Thurlow, Lazarus, Albus, & Hodgson, 2010), just like the technicians (as technological 
staff members). Other than that, the supervisors should also know what actions to do and how 
to do them as CBNE supervisors. 


Conclusions 


This research concludes that various constraints still occur in the implementation of 
CBNE in Indonesia. These constraints include problems in providing facilities such as 
the stable electric current, computers and rooms which meet the test standard/requirement; 
Internet network problems; the preparedness of CBNE participants in using the computer; 
and constraints related to the committee and supervisors. However, such constraints can be 
anticipated and overcome by means of several activities such as preparing, managing, and 
arranging strategies in maximizing the use of available facilities and resources at the school 
(for example, computers, servers, electricity, and networks); arranging the layout number of 
computers according to the room space available; anticipating and preparing students’ mental 
condition in facing the CBNE; making a solid collaboration between technicians and test 
supervisors; and giving training and briefing sessions before administering the CBNE. 

Considering the constraints in implementing the CBNE, some suggestions are made to 
deal with the challenges. First, there should be initiatives from the teachers to meet the standards 
of the equipment used to administer the CBNE. Second, the government should improve a more 
even distribution of facilities for implementing the CBNE such as electric power, networks, and 
computer sets to regions all over Indonesia. Third, ICT should be integrated in the curriculum 
as a school subject at the JHS and SHS levels to help the students prepare for the CBNE. 
Fourth, the students’ parents together with the society should be actively engaged in supporting 
the CBNE implementation. At last, schools in other countries which want to implement the 
CBNE need to understand the challenges and readiness of its components. This is to ensure 
that the schools can develop an appropriate strategy in order the CBNE implementation can run 
effectively and yield a good result without showing any bias. 
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